Is it possible to identify and measure perceptions and preferences for meritocracy as two distinct but related factors?
This research builds upon previous studies attempting to develop a comprehensive approach to measuring meritocratic beliefs (Duru-Bellat & Tenret, 2012; Kunovich & Slomczynski, 2007; Newman et al., 2015; Reynolds & Xian, 2014). The proposed measurement model is based on two axis of analysis, as depicted in Table 1. The first one distinguishes between different types of “beliefs”, using instead the terms perceptions and preferences for meritocracy. Perceptions refers to the extent to which people see meritocracy working in their society, which in terms of measurement relates to the “reasons to get ahead” battery, whereas preferences refer to normative expectations that are usually linked to a “should” expression (e.g. whether hard work should be related to payment). The second axis consider the distinction between meritocratic and non-meritocratic dimensions. This aspect has been usually treated as different ends of a same continuum in part of the previous research, an assumption that requieres empirical scrutiny. These non-meritocratic elements usually refer to the use of personal contacts or family advantages to get ahead in life.
Table 1: Model of perceptions and preferences for meritocracy and non-meritocracy
| Perceptions | Preferences | |
|---|---|---|
| Meritocracy | ||
| Non-meritocracy |
Hypotheses
The scale will be included in an online three-wave panel survey study. This study covers among other things an experiment of the effect of information about poverty and inequality on opportunity beliefs. The scale will be presented in wave 1 and wave 3.
The data will come from a representative sample of the large cities of Chile (Gran Santiago, Valparaíso-Viña del Mar, Concepción and Antofagasta). In order to ensure representativeness, quotas for sex, education and age will be provided to the agency in charge of the online fieldwork.
The sample will have an initial number for the first wave of 2100 cases, which are expected to be at least 1800 in the second wave. From the second to the third wave it has been considered as “free fall”, from which we expect to retain about 1500 cases.
The sample size was calculated from power analysis that used information from a pilot study that was applied in June 2019. This pilot study consisted of an online experiment applied to 949 subjects, randomly separating participants to two stimuli and a placebo, leaving a third of the sample in each condition. From these data, the low power of a two-tailed test was calculated, considering an average treatment effect of 0.26, with a standard deviation of the result of the treatment group of 1.23 and a significance level of 0.05. Based on multiple comparison correlations (Bonferroni), we adjusted alpha for six tests (three results * two comparisons between conditions and placebo). The results indicated that the number of subjects exposed to each stimulus should be 580, so we will work with three samples of 600 cases, which gives us a total of 1800 cases for the second wave in which the experiment will be applied. To guarantee the above and in consideration of a high non-response rate, we will start with a sample of 2100 subjects.
Respondents will be invited to participate in the online survey under the incentive provided by the external company. Respondents have 4 business days to answer the survey, with a maximum of 20 minutes per survey. In order to achieve greater representativeness of the sample, the quota method will be used, that is, the program will only allow respondents with the required demographic characteristics. The quotas used were age, sex and education. It should be noted that the time between wave and wave is around 7 and 9 days.
The rule of detention is to reach the number of respondents indicated in the quotas. However, there may be modifications depending on the contingencies of the field work and the response rates.
All the variables used in the scale use the following question and answer options:
| Dimension | Factor | Statement (english) | Statement (spanish) |
|---|---|---|---|
| Perception | Meritocratic | Those who try harder get greater rewards than those who work less. | Quienes más se esfuerzan logran obtener mayores recompensas que quienes se esfuerzan menos. |
| Those who have more talent achieve greater rewards than those who have less talent. | Quienes poseen más talento logran obtener mayores recompensas que quienes poseen menos talento. | ||
| Non meritocratic | Those who have rich parents succeed. | Quienes tienen padres ricos logran salir adelante. | |
| Those who have good contacts succeed. | Quienes tienen buenos contactos logran salir adelante. | ||
| Preference | Meritocratic | Those who try harder should get greater rewards than those who work less. | Quienes más se esfuerzan deberían obtener mayores recompensas que quienes se esfuerzan menos. |
| Those who have more talent should get greater rewards than those who have less talent. | Quienes poseen más talento deberían obtener mayores recompensas que quienes poseen menos talento. | ||
| Non meritocratic | It’s fine that those with rich parents get ahead. | Está bien que quienes tienen padres ricos salgan adelante. | |
| It’s fine that those who have good contacts get ahead. | Está bien que quienes tienen buenos contactos salgan adelante. |
There will not be variable manipulation.
The investigation of the validity of this scale does not rely on indices but rather confirmatory factor analysis will be used to estimate the value of the latent variables underlying the indicators.
No special blinding is used for this study.
To test the items’ order effects we present the items in three different order conditions which will be assigned randomly. The conditions are the following:
To evaluate the hypotheses, confirmatory factor analysis (CFA) will be used as we rely on a theory regarding the underlying four factors. The method of estimation will be Weighted Least Squares Mean Variance Approximation (WLMSV) for categorical ordered indicators. For the analysis we will use the R package Lavaan (Rosseel, 2012).
The values that will be used as evaluation criteria for the goodness of the adjustment of the model were taken from the proposal of Brown (2008) and are the following:
To evaluate the metric stability of the measurement model (Davidov et al., 2014) we will conduct a longitudinal invariance test using the first and third wave of our study. Following Liu et al. (2017), we will test a series of four hierarchical models: Configural, Weak, Strong and Strict invariance models for ordered-categorical indicators based on the assumption that a five category likert scale cannot be treated as a continous variable because can lead to biased parameter estimates.
All cases will be used as long as they do not show missing values in any of the items on the scale. No imputation criteria will be used.
The experiment and survey instruments are approved by the IRB of the University of Chile.
Brown, T. A. (2008). Confirmatory Factor Analysis for Applied Research. https://doi.org/10.1198/tas.2008.s98
Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement Equivalence in Cross-National Research. Annual Review of Sociology, 40(1), 55–75. https://doi.org/10.1146/annurev-soc-071913-043137
Duru-Bellat, A. M., & Tenret, E. (2012). Who’s for Meritocracy? Individual and Contextual Variations in the Faith. Comparative Education Review, 56(2), 223–247.
Kunovich, S., & Slomczynski, K. M. (2007). Systems of distribution and a sense of equity: A multilevel analysis of meritocratic attitudes in post-industrial societies. European Sociological Review, 23(5), 649–663. https://doi.org/10.1093/esr/jcm026
Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506. https://doi.org/10.1037/met0000075
Newman, B. J., Johnston, C. D., & Lown, P. L. (2015). False Consciousness or Class Awareness? Local Income Inequality, Personal Economic Position, and Belief in American Meritocracy. American Journal of Political Science, 59(2), 326–340. https://doi.org/10.1111/ajps.12153
Reynolds, J., & Xian, H. (2014). Perceptions of meritocracy in the land of opportunity. Research in Social Stratification and Mobility, 36, 121–137. https://doi.org/10.1016/j.rssm.2014.03.001
Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48(2), 1–36.